HEVC is an advanced video coding system developed under the Joint Collaborative Team on Video Coding (JCT-VC) group of video coding experts from ITU-T Study Group (High Efficiency Video Coding, Recommendation ITU-T H.265, Series H: Audiovisual And Multimedia Systems, Infrastructure of Audiovisual Services—Coding of Moving Video, International Telecommunication Unit, April, 2013). In HEVC, one slice is partitioned into multiple coding tree units (CTU). For color video data, each CTU consists of multiple coding tree blocks corresponding to the color components. In main profile, the minimum and the maximum sizes of CTU are specified by the syntax elements in the sequence parameter set (SPS) among the sizes of 8×8, 16×16, 32×32, and 64×64. For each slice, a raster scan traversing through the slice is used for processing the CTU.
The CTU is further partitioned into multiple coding units (CU) to adapt to various local characteristics. A quadtree denoted as the coding tree is used to partition the CTU into multiple CUs. Let CTU size be M×M where M is one of the values of 64, 32, or 16. The CTU can be a single CU or can be split into four smaller units of equal sizes of M/2×M/2, which are nodes of coding tree. If units are leaf nodes of coding tree, the units become CUs. Otherwise, the quadtree splitting process can be iterated until the size for a node reaches an allowed minimum CU size specified in the SPS (sequence parameter set).
FIG. 1A illustrates an example of partitioning result for a CU using the quadtree partition process recursively. Every time when a block is partitioned by a quadtree, the original block is split into four sub-blocks. In the next level, the sub-block becomes a new block to be further partitioned. The partitioning process may decide not to split a block. In this case, the block is not further partitioned. The partitioning process may be terminated when a minimum quadtree block size is reached. In some cases, the partitioning process may be terminated when the partition depth reaches a maximum value. The final sub-blocks in solid lines having various block sizes as shown in FIG. 1A correspond to the boundaries of coding units generated from the partitioning process. This partition process results in a recursive structure representing the partition decisions as shown in FIG. 1B and the tree-like structure in solid lines is called a coding tree, where each leaf node corresponds to a CU (i.e., one final sub-block). The decision whether to code a picture area using inter-picture (temporal) or intra-picture (spatial) prediction is made at the CU level. In HEVC, the minimum CU size can be 8×8. Therefore, the minimum granularity for switching different prediction type is 8×8.
For prediction process (e.g. inter prediction or intra prediction), each CU is further partitioned into one or more prediction units (PUs). Coupled with the CU, the PU works as a basic representative block for sharing the prediction information. Inside one PU, the same prediction process is applied and the relevant information is transmitted to the decoder on a PU basis. A CU can be split into one, two or four PUs according to the PU splitting type. HEVC defines eight shapes for splitting a CU into one or more PUs as shown in FIG. 2. Unlike the CU, the PU may be split only once. In FIG. 2, the lower four partitions correspond to asymmetric partition.
After obtaining the residual block for a CU by applying the prediction process to the one or more PUs generated by the splitting process, a CU can be partitioned into transform units (TUs) according to another quadtree structure similar to the coding tree for the CU. In FIG. 1A, the dotted lines indicate the resulting TU boundaries by quadtree partition of each CU. The TU is a basic representative block having residual or transform coefficients for applying the integer transform and quantization. For each TU, one integer transform having the same size as the TU is applied to obtain residual coefficients. These coefficients are transmitted to the decoder after quantization on a TU basis.
The terms, coding tree block (CTB), coding block (CB), prediction block (PB), and transform block (TB) are defined to specify the 2-D sample array of one color component associated with CTU, CU, PU, and TU, respectively. Thus, a CTU consists of one luma CTB, two chroma CTBs, and associated syntax elements. A similar relationship is valid for CU, PU, and TU.
The same tree partitioning is generally applied to both luma and chroma components, although exceptions may apply when certain minimum sizes are reached for chroma.
The current HEVC block partitioning only uses the quadtree based partitioning to partition a CTU to CU and to partition a CU to TU in a recursive fashion until a limit is reached. On the other hand, the current HEVC allowed up to 8 partition types for the PU. However, the PU partition is only performed once for each PU. Therefore, it is desirable to further improve the coding efficiency to meet the needs of ever increasing storage and transmission of video contents.